test(e2e): harden Appium Test Dapp readiness for confirmations smokes (MMQA-2254) by chrisleewilcox · Pull Request #34577 · MetaMask/metamask-mobile

chrisleewilcox · 2026-08-11T00:19:28Z

Description

appium-confirmations-android-smoke was consistently failing across unrelated PRs (shards 1 and 2), blocking merges.

Root cause is Test Dapp readiness / Appium helper timing — not a product regression:

Appium waitForTestDappToLoad was too weak (URL bar only). Action buttons stay disabled until page JS + provider connection complete.
After navigate/reload, the Test Dapp often loses the EIP-6963 Active Provider race (initialize() checks providerDetails.length before announce arrives). Active Provider UUID/Name stay blank, so Connect uses an unset globalContext.provider while window.ethereum still answers eth_accounts.
Android confirmation taps were re-issued every ~3s while waiting for confirm-button, thrashing in-flight eth_sendTransaction (Test Dapp showed "Creation Failed").
iOS WebView readiness waits must return to NATIVE_APP so later native taps (e.g. close browser) still work.

This PR hardens the shared helpers used by confirmations Appium smokes:

Stronger Appium Test Dapp load waits (Android native chrome; iOS WebView logo/title + always reset to native context)
Android-only hydration: click EIP-6963 "Use MetaMask", then Connect (approve sheet only if shown), wait for #accounts
Wait for DOM-enabled target before tap
Longer per-tap wait for the confirmation sheet (15s) with sparse retries + push-notification dismiss
preferNative option for Android WebView taps (unit-tested)

Changelog

CHANGELOG entry: null

Related issues

Fixes: https://consensyssoftware.atlassian.net/browse/MMQA-2254

Related: https://consensyssoftware.atlassian.net/browse/MMQA-2232

Manual testing steps

Feature: Confirmations Appium smoke stability

  Scenario: Android confirmations smokes open confirmation sheets from Test Dapp
    Given a main-e2e Android build and emulator
    And fixtures with the test dapp connected on Anvil
    When appium-confirmations-android-smoke runs
    Then shard 1 can enable/tap Test Dapp actions (sendEIP1559 / createToken / eip5792SendCalls)
    And shard 2 approve / setApprovalForAll flows open confirm-button

  Scenario: iOS confirmations smokes remain green
    When appium-confirmations-ios-smoke runs
    Then signature and transaction confirmation flows still pass

CI validation on this PR (4a43fc9718e):

appium-confirmations-android-smoke 1 & 2: pass
appium-confirmations-ios-smoke 1 & 2: pass
Full CI run: https://github.com/MetaMask/metamask-mobile/actions/runs/31464720887

Local unit coverage:

yarn jest tests/framework/AndroidWebViewNative.test.ts --watchman=false

Screenshots/Recordings

N/A — test/infra-only change; no product UI changes.

Before

N/A

After

N/A

Pre-merge author checklist

I've followed MetaMask Contributor Docs and MetaMask Mobile Coding Standards.
I've completed the PR template to the best of my ability
I've included tests if applicable
I've documented my code using JSDoc format if applicable
I've applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Performance checks (if applicable)

I've tested on Android
- Ideally on a mid-range device; emulator is acceptable
I've tested with a power user scenario
I've instrumented key operations with Sentry traces for production performance metrics

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
Test-infra only: changes Appium E2E helpers and selectors with no product, auth, or data-path impact.

Overview
Stabilizes flaky Appium confirmations smokes by making Test Dapp readiness waits match real interactivity, not just URL-bar presence.

waitForTestDappToLoad now waits for page chrome on Appium (Android native title/WebView; iOS logo/title) and always resets to native context on iOS so later native taps keep working.

Confirmation taps hydrate the provider before acting: Android selects EIP-6963 Active Provider, Connects if needed, waits for #accounts/enabled buttons (re-hydrating after contract reload), then uses a longer 15s post-tap confirm wait with sparse retries. iOS waits for provider readiness and an enabled control. Adds a unit-tested preferNative WebView tap option so retries can skip false-success CDP clicks.

^{Reviewed by Cursor Bugbot for commit 418aef7. Bugbot is set up for automated code reviews on this repo. Configure here.}

… (MMQA-2254) Wait for page chrome, provider injection, and enabled controls before tapping; prefer native Android WebView taps so CDP false-success clicks do not block CI. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions · 2026-08-11T00:19:37Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

metamask-ci · 2026-08-11T00:20:23Z

PR template — items to address before "Ready for review"

Warnings — informational, address before merging:

Pre-merge author checklist has unchecked items (e.g. "I've applied the right labels on the PR (see labeling guidelines). Not required for external contributors."). Every box must be consciously checked — see docs/readme/ready-for-review.md.

See docs/readme/ready-for-review.md for the full Definition of Ready for Review.

iOS WebView logo/title waits left the session in WEBVIEW, breaking native close-browser taps in network-manager2. Use native WebView + title text instead. Co-authored-by: Cursor <cursoragent@cursor.com>

… reset (MMQA-2254) iOS does not expose the Test Dapp title in the native tree. Keep WebView logo/title waits on iOS and always switch back to NATIVE_APP afterward. Co-authored-by: Cursor <cursoragent@cursor.com>

…QA-2254) After reload, fixture permissions can inject ethereum without globalConnectionChange, leaving action buttons disabled. Request accounts, emit the dapp connection event, then use trusted CDP clicks with native fallback so confirm sheets open reliably on Android Appium. Co-authored-by: Cursor <cursoragent@cursor.com>

…s (MMQA-2254) Screenshots showed Create Token already clicked ("Creation Failed") while CI re-tapped every 3s waiting for confirm-button. Wait 15s per tap for gas estimation, nudge dapp UI gates without eth_requestAccounts, and emit blockBaseFeePerGasUpdate so EIP-1559 send enables. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

codecov-commenter · 2026-08-11T02:57:44Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.11%. Comparing base (cbe5e23) to head (0e53415).
⚠️ Report is 17 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #34577      +/-   ##
==========================================
- Coverage   85.12%   85.11%   -0.01%     
==========================================
  Files        6350     6369      +19     
  Lines      173080   173713     +633     
  Branches    42804    42977     +173     
==========================================
+ Hits       147333   147864     +531     
- Misses      15708    15771      +63     
- Partials    10039    10078      +39

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…s (MMQA-2254) CI screenshots showed NOT CONNECTED / Account undefined after we dispatched globalConnectionChange — buttons enabled without src.provider. Click Connect so the dapp runs its real eth_requestAccounts handler, then wait for #accounts. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

…MMQA-2254) CI showed Active Provider UUID/Name empty while eth_accounts still worked — Connect used unset globalContext.provider. Click "Use MetaMask" after the EIP-6963 button appears, then hydrate accounts via Connect as before. Co-authored-by: Cursor <cursoragent@cursor.com>

…2254) iOS evaluateInWebView cannot await async eth_accounts Promises, so the new hydration path timed out and broke previously-green iOS confirmations smokes. Keep EIP-6963 Active Provider + Connect hydration on Android only. Co-authored-by: Cursor <cursoragent@cursor.com>

Resolve confirmations.flow.ts by keeping Android EIP-6963 hydration and 15s confirm waits with main's waitForTestDappButtonReady contract binding. Co-authored-by: Cursor <cursoragent@cursor.com>

…254) waitForTestDappButtonReady can location.reload() on ?contract= pages, which clears Active Provider and accounts. Re-hydrate and wait again before tapping so approve flows do not hit the EIP-6963 race on the same attempt. Co-authored-by: Cursor <cursoragent@cursor.com>

chrisleewilcox · 2026-08-11T16:24:45Z

Addressed Bugbot finding (hydration lost after contract reload) in c52a928: waitForTestDappButtonReady now reports whether it reloaded, and on Android we re-run EIP-6963 account hydration + button readiness before tapping so a contract-binding reload cannot leave Active Provider / #accounts wiped on the same attempt.

github-actions · 2026-08-11T21:27:20Z

🔍 Smart E2E Test Selection

Selected E2E tags: SmokeNetworkExpansion, SmokeNetworkAbstractions, SmokeConfirmations, SmokeWalletPlatform, SmokeBrowser, SmokeMultiChainAPI, SmokeSnaps
Selected Performance tags: None (no tests recommended)
Risk Level: medium
AI Confidence: 92%

click to see 🤖 AI reasoning details

E2E Test Selection:
Shared test infra changed (tests/flows/browser.flow.ts, tests/flows/confirmations.flow.ts, tests/selectors/Browser/TestDapp.selectors.ts). Found 58 affected spec file(s). Running tags: SmokeNetworkExpansion, SmokeNetworkAbstractions, SmokeConfirmations, SmokeWalletPlatform, SmokeBrowser, SmokeMultiChainAPI, SmokeSnaps

Performance Test Selection:
No app source code was changed in this PR. All changes are to E2E test infrastructure files (flows, framework utilities, selectors). These changes do not affect app runtime performance, rendering, or any measured user flows. No performance tests are warranted.

View GitHub Actions results

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 418aef7. Configure here.}

cursor · 2026-08-11T21:28:49Z

+          timeout: ANDROID_CONFIRM_SHEET_TIMEOUT_MS,
+          // Keep retries sparse so we do not stampede eth_sendTransaction.
+          interval: 2_000,
+        },


Retry budget shorter than hydration

Medium Severity

Android confirm retries use a 60s executeWithRetry budget, but each attempt can spend up to 60s in ensureTestDappAccountsHydrated alone (30s active-provider wait plus 30s accounts wait) before the 20s button wait and 15s confirm wait. Slow hydration can exhaust the outer timeout with no remaining confirm retries, which undercuts the sparse re-tap design.

Additional Locations (2)

tests/flows/confirmations.flow.ts#L32-L37

tests/flows/confirmations.flow.ts#L206-L262

^{Triggered by project rule: MetaMask Mobile E2E Testing Guidelines}

^{Reviewed by Cursor Bugbot for commit 418aef7. Configure here.}

sonarqubecloud · 2026-08-11T21:47:38Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

metamask-ci Bot added the team-qa QA team label Aug 11, 2026

github-actions Bot added the size-M label Aug 11, 2026

chrisleewilcox and others added 5 commits August 10, 2026 17:35

test(e2e): keep Appium Test Dapp readiness in native context (MMQA-2254)

5d70dd3

iOS WebView logo/title waits left the session in WEBVIEW, breaking native close-browser taps in network-manager2. Use native WebView + title text instead. Co-authored-by: Cursor <cursoragent@cursor.com>

test(e2e): fix prettier wrap in confirmations flow (MMQA-2254)

0e53415

Co-authored-by: Cursor <cursoragent@cursor.com>

chrisleewilcox and others added 2 commits August 10, 2026 20:27

test(e2e): use interface for Test Dapp connection state (MMQA-2254)

ac03fe7

Co-authored-by: Cursor <cursoragent@cursor.com>

chrisleewilcox mentioned this pull request Aug 11, 2026

test(e2e): skip consistently failing Appium confirmations Android smokes (MMQA-2254) #34586

Merged

10 tasks

chrisleewilcox and others added 2 commits August 10, 2026 21:07

chrisleewilcox marked this pull request as ready for review August 11, 2026 14:09

chrisleewilcox requested a review from a team as a code owner August 11, 2026 14:09

Merge origin/main into MMQA-2254-appium-confirmations-dapp

0b66b82

Resolve confirmations.flow.ts by keeping Android EIP-6963 hydration and 15s confirm waits with main's waitForTestDappButtonReady contract binding. Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions Bot added the risk:low AI analysis: low risk label Aug 11, 2026

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread tests/flows/confirmations.flow.ts

Merge branch 'main' into MMQA-2254-appium-confirmations-dapp

418aef7

cursor Bot reviewed Aug 11, 2026

View reviewed changes

Navigation Menu

Uh oh!

Uh oh!

test(e2e): harden Appium Test Dapp readiness for confirmations smokes (MMQA-2254) - #34577

test(e2e): harden Appium Test Dapp readiness for confirmations smokes (MMQA-2254)#34577
chrisleewilcox wants to merge 13 commits into
mainfrom
MMQA-2254-appium-confirmations-dapp

chrisleewilcox commented Aug 11, 2026 •

edited by cursor Bot

Loading

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

metamask-ci Bot commented Aug 11, 2026

Uh oh!

codecov-commenter commented Aug 11, 2026

Uh oh!

Uh oh!

chrisleewilcox commented Aug 11, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Aug 11, 2026

Uh oh!

sonarqubecloud Bot commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

chrisleewilcox commented Aug 11, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changelog

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

Performance checks (if applicable)

Pre-merge reviewer checklist

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

metamask-ci Bot commented Aug 11, 2026

PR template — items to address before "Ready for review"

Uh oh!

codecov-commenter commented Aug 11, 2026

Codecov Report

Uh oh!

Uh oh!

chrisleewilcox commented Aug 11, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

🔍 Smart E2E Test Selection

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 11, 2026

Choose a reason for hiding this comment

Retry budget shorter than hydration

Uh oh!

sonarqubecloud Bot commented Aug 11, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chrisleewilcox commented Aug 11, 2026 •

edited by cursor Bot

Loading